It has become common to develop computing applications for computing environments that include a variety of computing servers and computing clients. In particular, it has become common for a particular computing application to be developed to function suitably with a wide variety of computing clients. For example, so-called Web and/or JavaScript® applications are commonly expected to operate with a wide variety of hypertext document browser clients including browsers developed by multiple vendors for multiple different hardware platforms and having multiple different versions each with different capabilities. This is a challenging development environment in which application testing can play an even more significant role than usual. Several conventional testing suites have been developed for such environments, and particularly for testing Web applications. However, each such testing suite has its shortcomings.
For example, some conventional test suites are unreliable, inconsistent, inefficient and/or ineffective. Failure to deal properly with ephemeral, transitory and/or race conditions can result in a high rate of false negative (e.g., indications that a test has been failed when a client is actually behaving as desired) and/or false positive (e.g., indications that a test has been passed when the client is actually misbehaving) test results. Since this is typically unacceptable, test developers may undertake development of “compensation code” to attempt to correct a conventional test suite's failure to deal properly with certain conditions. However, such compensation code tends to be complex and/or client-specific and can require expert knowledge of each of the plurality of clients. The complexity alone is problematic, since it can raise the likelihood that the testing process itself introduces errors (sometimes called computer programming “bugs”).
As another example, some conventional test suites delegate test object identification to a client under test. This can open another avenue for unreliability and/or inconsistency since the client's object identification mechanism may be flawed and/or ambiguous, again raising the likelihood of false negative and/or false positive test results. In such cases, even the stop-gap of compensation code may not be available to test developers.